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ABSTRACT 



y^paratus is described for processing a digitized image 
signal obtained from an image sensor having color photo- 
sites aligned in rows and colmnns that generate at least three 
separate color values but only one color value for each 
photosite location, structure for interpolating color values 
for each photosite location so that it has three different color 
values. The apparatus includes a memory for storing the 
digitized image signal and a processor operative with the 
memory for generating an appropriate color value missing 
from a photosite location by the interpolation of an addi- 
tional color value for such photosite locations from color 
values of different colors than the missing color value at 
nearby photosite locations. The processor also includes 
structure for obtaining Laplacian second-order values and 
gradient values in at least two image directions from nearby 
photosites of the same colimm and row and for adding the 
Laplacian second-order values and the gradient values to 
define a classifier and for selecting a preferred orientation 
for the interpolation of the missing color value based upon 
a classifier. Finally, an arrangement is provided for interpo- 
lating the missing color value from nearby multiple color 
values selected to agree with the preferred orientaticn. 

14 Claims, 5 Drawing Sheets 
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ADAPTIVE COLOR PLAN INTERPOLATION means for obtaining Laplacian second-order values and 

IN SINGLE SENSOR COLOR ELECTRONIC gradient values in at least two image directions from 

CAMERA nearby pihotosites of the same column and row; 

„ ^ means for adding die Laplacian second-order values and 

CROSS RErajENOS TO REl^D 5 the gradient values to define a classifier and for select- 

AEPLECAriONS ^^g a preferred orientation for the interpolation of the 

Reference is made to commonly assigned U.S. Ser. No. missing color value based upon a classifier, and 

08/407,436 filed Mar. 17, 1995, issued as U.S. Pat. No. means for interpolating the missing color value from 

5,506,619 on Apr. 9, 1996 to James E. Adams, Ir. and John nearby multiple color values selected to agree with the 

F. Hamilton, Jr. filed concucrently herewith, the teachings of preferred orientation. 



which are incorporated harein. 

FIELD OF THE INVENTION 



Advantages 



The advantages of this invoition are 1) is computationally 
This invention relates to the field of electronic imaging efficient both in execution time and memory storage rcquire- 
and is partlailarly suitable to elccu-onic still imaging by mcnts; and 2) by use of the combination of die Upladan 
means of an electronic still camera having a single colar second-order values and the gradient values to produce a 
sensor and memory. classifier, artifacts (color interpolation) in ou^t image are 

BACKGROUND OF THE INVENTION substantially reduced 

In electronic color imaging, it is desirable to simulta- ^ BRIEF DESCRimON OF THE DRAWINGS 
neously capture image data in tiirce color planes, usuaUy no. lis a block diagram of an electronic stiU camera 
red, green and blue. When the three color planes are employing interpolation processing according to the inven- 
combincd, it is possible to create high-quality color images. ^j^^. 

Capturing these three sets of image data can be done in a ^- __ ^ . . , , j.^, , . * 1 

tT^ * T 1 . ■ u.. *ui. .^rr.^ ^ FIG.2isablodcdiagramofthelogicof themterpolation 

number of ways. In electronic photography, this is some- * i_ - . ^ 

uuiu^ w / ^ J •„« «™, processing techmque used m connection with the mvention; 

tunes accomphshcd by usmg a smglc two dimensional array *^ ^ ^ ^, , . 

of sensors that are covered by a pattern of red, green and FIG. 3 is a detaUed block diagram of the logic of the 
blue filters. This type of sensor is known as a color filter interpolation luminance section m FIG. 2: 
array or CPA. Below is shown the red (R), green (G) and FIG. 4 is a more detailed blockdiagram of the logic of the 
blue (B) pixels as arc commonly arranged on a CPA sensor. chrominance section in FIG. 2; and 

When a color image is c^tured using a CPA, it is FIGS. 5 and 6 show logic block diagrams for the inter- 
necessary to interpolate the red, green and blue values so polation processes in accordance with the invention. 

that there is an estimate of all three color values for each ^„ 

sensor location. Once the interpolation is done, each picture 35 DETAILED DKCTffTEON OF THE 

element, or pixel, has three color values and can be pro- INVEmlON 
cessed by a variety of known image processing techniques since single-sensor electronic cameras enjoying color 
depending on the needs of the system. Some exana$>les of the filter arrays are well known, the present description will be 
reasons for processing are to do image sharpening, color directed in particular to elements forming part of, or coop- 
conecdon or halftoning. 40 crating more directly with, apparatus and method in accor- 

The following shows how red green and blue pixels can dance with the present invention. Elements not specifically 
be arranged in a color filter array. For a more detailed shown or described herein may be selected from those 
descripdon see U.S. PaL No. 3,971,065 to Bayer. known in the art 

Referring initially to FIGS. 1 and 2, an electronic still 
OR OR 45 camera is divided generally into an input section 2 and. an 

^ °^ ^ interpolation and recording se<^on 4. The input section 2 

Q includes an exposure section 10 for directing image light 

from a subject (not shown) toward an image sensor 12. 
Although not shown, the exposure section 10 includes 
SUMMARY OP INVENTION 50 conventional optics for directing the image light through a 

diaphragm, which regulates the optical apertm'e, and a 
The object of this invention is to provide an improved ^^^^^ regulates exposure time. The image sensor 12, 
apparatus for estimating the missing pixel values in a CPA. includes a two-dimensional array of photosites cor- 

This object is adiieved in apparatus for processing a responding to picture elements of the image, is a conven- 
digitized image signal obtained from an image sensor having 55 tional charge-coupled device (CCD) using either well- 
color photosites aligned in rows and columns that generate known interline transfer or frame transfer techniques. The 
at least three separate color values but only one color value i^age sensor 12 is covered by a color filter array (CPA) 13, 
for each photositc location, means for interpolating color known as the Bayer array, which is described b U.S. Pat 
values for each photositc location so tiiat it has three j^o. 3,971,065 and herewith incorporated by reference. In 
different color values conaprising: go the Bayer geometry each color covers aphotosite, or picture 

means for storing the digitized image signal; element (pixel), of the sensor. In particular, chrominance 

a processor operative with said storing means for gener- colors (red and blue) are interspersed among a checkerboard 
ating an appropriate color value missing from a pho- pattern of luminance colors (green). The image sensor 12 is 
tositc location by the interpolation of an additional exposed to image light so that analog image charge infor- 
color value for such photosite locations firom color 65 mation is generated in respective photosites. The charge 
values of different colors than the missing color value information is applied to an output diode 14, which converts 
at nearby photositc locations, said processor including the charge information to analog image signals correspond- 
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ing to respective picture elements. The analog image signals An operation display panel 30 is connected to the control 

are applied to an A/D converter 16, which generates a digital processor 20 for displaying information useful in operation 

image signal from the analog input signal for each picture of the camera. Such infcmation might include typical 

element The digital signals are applied to an image buffer photographic data, such as shutter speed, aperture, exposure 

18, which may be a random access memory (RAM) with 3 bias, color balance (auto, tungsten, fluorescent, dayUght), 

storage c^aaty for a pluraUty of still images. field/frame, low battery, low Ught. exposure modes (aperture 

Acontrolprocessor20generallycontrolstheinputsection preferred, shutter preferred), and so on. Moreover, other 

2 of the camera by initiating and controlling exposure (by infonnation unique to this type of camera is displayed. For 

operaUon by the diaphragm and shutter (not shown) in die ^stmc^^ the removable memory card 24 would ordinarily 

exposure section 10) by generating the horizontal and include a directory signifying the beginning and ending of 

vertical docks needed for dnving the image sensor 12 and stored image. This would show on the display panel 30 

for dockmg image mformauon therefrom, aud by enabhng ^he number of images stored or the 

&eA/Dconvcrtcrl6mconjunctionwiththeimagebufferl8 ^^^r of image spaces remainingT^ estimated to be 

for each signal segment relatmg to a picture dement (The remaining. 

control processor 20 would ordinarily indudc a micropro- , «^ . . , .„ 

cesser coupled with a system timing drcuit.) Once a ceiiain .7^^. ^^^^ 22 mtopolates eadi stdl 

number of digital image signals have been accumulated in ° "f^P stored m the miage buffer 18 accordmg to the 

theimagebuffcrl8,thestoredsignalsareappliedtoadigital interpolaUon techmque shown in FIG 2. Ihc mtapolaUon 

signal processor 22, which controU the throughput pro^s- ^^Z ^ valuK at each pixd lo^on foUows the 

ingratefortheinterpolationandrccardings^tion4ofthe sequence shown m HG. 2; that is fost, ft^^^^ 
camera. TTie digital signal processor 22 aK>Ues an interpo- '° °°^tion for the "missmg green" pixds (i,e., the red and 

lation algorithm to the digital image signals, and sends Ihe "'^'^ pixd locations) are intapolatcd to i^ve the himi- 

inteipolated signals to a conventional, removable memory ^.ai^^e ren(htion and, secondly the color difference mforma- 

card 24 via a connector 26 ^ Uon is intapolated at the high frequency locations by 

o:„« i^t t„*,-«« «1j w J . J- 1 bilinear methods to generate die other colors of the CFA 

Smcc the interpolation and related processmg ordinanly 55 t *u * 1 t. • . . 

, ^ ..... T ^ . '^^ pattern. In the implementabon shown in FIG. 2, an adaptive 

occurs over several steps, the intermediate products of the . ^ , . . wu ^ ^ *w. ou ai«^iAYc 

, ' J . « VI mterpolation techmque is used m the luminance section 36 

processmg algonthm are stored in a processing buffer 28. fuTI^™ !^ lu j^ciuimiwuuc ^ccuuu 

/rm. ' tt. ^ J ^ lor optimizing the performance of the system for imaees 

(The processing buffer 28 may also be configured as part of .*u 1. • * 1 j 1 ^ uu^s^a 

die nLory space of the image buffer 18.) THe nmiS>er of ^"Tf^^ ?^ ""f^ "^"l ^? ^^^^ 

image signVneeded in the iLge buffer 18 before digital 30 r.Sn"?^ "^fT^^^^ 

processing can begin depends onL type of processing,^at S^^^'^T^^ depending upon Uje gradient estabhshed 

u w , n^;«hK«fhr^«H i„f^r„«i,ri^« Jl TIt^SJiT^ between the durominance (red and blue) pixel locations in 

IS, for a nci^borhood interpolation to begm, a block of ^ ^ horizontal directions around the -^sing 

signals mdudmg at least a portion of the image signals green" oix L ^^^Jua oiuuuu uic imi^mg 

con^rising a video frame must be available. Consequently, ^ 

in most circumstances, the interpolation may commence as 55 adaptively interpolating the 'Wssing 

soon as the requisite block of picture elements is present in ^reen" pixels is to select an interpolation method. The 

the buffer 18. details of this process are shown in block 40 of FIG. 3. The 

The input section 2 operates at arate commensurate with P^f^s starts by computing two composite pixd dassifier 
normal operation of the camera whUe interpolation, whidi ^^^f !2l ^ ' honzontel direction and one 
may consumemoretime, can be relatively divorcedfromthe 40 ^^r the verticaL The term ^uel classifier^' denotes a value 
inputrate.The exposure section 10 exposes the image sensor computed for the purpose of making adedsion aboutfurther 
12 to image light for a time period dependent upon exposure F^cessmg of the pad information. TTie term "composite- 
requirements, for example, a time period between Viooo ^'^P^^*^ dependency of the value on a multipHcity of 
second and several seconds. The image diarge is then swept c^Iot Planes. In diis case, the absolute value of the Uplacian 
from the photosites in the image sensor 12, converted to a 45 green plane is added to the absolute value of the 
digital format and written into the image buffer 18. The °^ ^^"^ P^^' depending on which was 
driving signals provided by the control processor 20 to the correspondmg cdor in the Bayer color filter array, 
image sensor 12, the AID converter 16 and the buffer 18 are ^ classifier values are the compared (Block 52) and 
accordingly generated to achieve such a transfer. TTie pro- tested for equality. In the likely event that one value is 
cessing throughput rate of the interpolation and recording 50 s™^cr than the other, the interpolation method correspond- 
section 4 is determined by the speed of the digital signal ^Z *e smaller value is selected (Block 54). If the values 
processor 22. are equal, then the default interpolation method is sdected 

One desirable consequence of this architecture is that the (^^^^ 56). In cither case Block 40 is done, 

processing algorithm employed in the interpolation and The green (luma) interpolation step (Block 44) has two 

recording section may be sdected for quality treatment of 55 P^s» as shown in FIG. 4. The first part (Block 80) averages 

the image rather than for throughput speed. This, of course, the two luminance (green) values according to the sdected 

can put a dday between consecutive pictures which may interpolation method. The second part (Block 82) adds a 

affect the user, depending on the time between photographic correction factor based on either red or blue ndghboring 

events. This is a problem since it is well known and values depending on if the pixd in question was covered by 

understood in the fidd of dectronic imaging that a digital 60 ^ red or blue filter in the Bayer color filter array, 

still camera should provide a continuous shooting capability The red/blue (chroma) interpolation proceeds in a manner 

for a successive sequence of images. For this reason, the similar to the green (luma) interpolation described above, 

image buffer 18 shown in FIG. 1 provides for storage of a The details of this process arc shown in Block 46 of FIG. 5. 

plurality of images, in effect allowing a scries of images to The process starts by computing two composite pixel clas- 

"stackup" atvideorates.Thesizeof the bufferis established 65 sificr values (Block 60), one for the negative diagonal 

to hold enough consecutive images to cover most picture- direction and one for the positive diagonal The term **nega- 

taking situations. tive diagonal" denotes the line of slope -1 (connecting tfie 
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upper left to the lower right). The term '^positive diagonal" 
denotes the line of slope +1 (connecting the lower left to the 
upper light). Again, these composite classifiers are found by 
adding the absolute value of the Lapladan in the green plane 
to the absolute value of the gradient in either the red or blue 
plane, depending on which color is being interpolated. 

The two classifier values are then compared (Block 62) 
and tested for equality. In the likely event that one value is 
smaller &an the other, the interpolation method correspond- 
ing to the smaller value is selected (Block 64). If the values 
are equals then the default inteipolation method is selected 
(Block 66). In either case Block 46 is done. 

The red/bhie (chroma) inteq)olation step (Block 4$) has 
two parts, as shown in FIG. 6. Id the first part (Block 70) two 
chrominance values, either red or blue depending on the 
pixel's position in the Bayer color filter airay, are averaged 
according to the selected interpolation method. The second 
part (Block 72) adds a coirectioa factor based on green 
neighboring values. 

More specifically, die following is a d^ailed descripdon 
of the operation of the digital signal processor for a specific 
exan^)le using the Bayer array. 

Green Plane Interpolation 

The first pass of the interpolation fully populates the green 
color plane. The Bayer color filter array is assumed. Con- 
sider the following neighborhood. 

Al 
G2 

A3G4A5G6A7 
G8 
A9 

Gx is a green pixel and Ax is either a red pixel or a blue 
pixel. (All Ax pixels will be the same color for the entire 
neighborhood.) For simplicity, we will use the term 
"chroma" to mean either red or blue. We form the following 
classifiers. 

DH=I-A3+2A5-A7WG4-G6I 
DV=l-Al+2A5-A9WG2-G8t 

These classifiers are conqK>sed of Laplacian second-order 
terms for the chroma data and gradients for the green data. 
As such, these classifiers are sensing the high spatial fre- 
quency information present in the pixel neighborhood in the 
horizontal (DH) and vertical (DV) directions. 

We then form three predictors. 

05H=(04+O6y2+<-Aaf2A5-A7y4 
05V=(Gi+08y2-K-Al+2A5-A9y4 
G5Ap<0^+<>W36ta8y44(-Al-A344A5-A7-A9y8 

These predictors are coiiqx3sed of arithmetic averages for 
the green data and appropdately scaled Laplacian second- 
order terms for the chroma data. G5H is to be used when the 
preferred orientation for the interpolation is in the horizontal 
direction within the pixel neighborhood Simihuiy, G5V is 
to be used when the preferred orientation for the interpola- 
tion is the vertical direction. G5A is used when there is no 
dear preference for orientation for the interpolation. 

The complete green interpolation process may now be 
expressed as bdow. 
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IFDH<DV 

'niENGSi=G5H 
ELSEIFDV<DH 

IHENGS^GSV 
5 ELSEG3 = G3A 

The key to this process is that both the green and tiic 
chroma data rmist indicate a minimmn of high spatial 
frequency information for a given orientation to be diosen 
as the preferred orientation for the interpolation. If there is 
a large amount of high spatial frequency information in 
either the green data or chroma data for a given orientation, 
it will inflate the value of the corresponding dassifier. This, 
in turn, reduces the likelihood for that orientation to be 
chosen as the preferred orientation for the interpolation. 

In practice, the green interpolation process may be 
simplified, as below, for computational ef&dency. 

IFDH<;=DV 
20 THENG5=.G5H 
ELSEG5 = G5V 

This simplification is achieved by defining the horizontal 
direction as the default preferred orientation for the inter- 
2^ polation when both horizontal and vertical dassifiers arc 
equal The number of occurrences in a typical image when 
the horizontal and vertical classifiers are equal is so small 
that this simplification generally has negligible impact on 
the image quality of the final teconstnicted image. 

30 

Red and Blue (Chroma) Interpolation 

The second pass of the interpolation fully populates the 
red and blue color planes. In U.S. Pat No. 4,642,578, issued 
Feb. 10, 1987, Cok (the disdosure of whidi is incorporated 
by reference herein) disdosed the chroma interpolation 
summarized below. Consider the following neighborhood. 

A1G2A3 
G4C3G6 
40 A7G8A9 

Gx is a green pixel, Axis cither a red or blue pixel and C5 
is the opposite color pixd to Ax (ic., if Ax is red then C5 
is bke and visa versa). Note that all Gx pixds, Gl through 
G9, are assumed to be known and coinddent with all 
corresponding Ax and C5 pixels. 

There are three cases. Case 1 is when the nearest ndgh- 
bors to Ax are in the same column. The following predictor 
5Q is used. (A4 is used as an example.) 

A4«<Al+A7yZ+<-Gl+204-G7y2 

Case 2 is when the nearest ndghbors to Ax are in the same 
row. The following predictor is used. (A2 is used as an 
55 exanq)le.) 

A2^Al+A3y2+<-<31+2G2-G3y2 

Case 3 is when the nearest neighbors to Ax are at the four 
coners. The following predictor is used. (AS is used as an 
^ exan^le.) 

A5=<A14A5+A7+A9y44<-Gl-G3+4G5-G7-G9y4 

We describe an improvement for case 3. We form the 
following two classifiers. 

65 

DN=l-Gl+205-<39WAl-A9I 
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DP=M33+2G5-G7M4A3-A7I 14 output (Uode 

16 AID converter 

These classifiers are composed of Laplacian second-order imaee buffer 
tenns for the green data and gradients for the chroma data. 20 ©onfrol nrocessor 
AS such, these classifiers are sensing the high spatial fi:e^ 5 22 digital signal processor 
quency information present in the pixel neighborhood in the 34 j-emovable memory card 
negative diagonal (DN) and positive diagonal (DP) direc- 26 connector 

^^°^* 28 processing buffer 

We then form three predictors. ^^^^^ p^^l 

A5N=(Ai+A9y24<^i+205-C39y2 ^° ^6 luminance section 

38 chroma sec^on 

A5P=<A3f A7y24<-<5J+2G5-G7y2 40 select best luma interpolation 

44 interpolate missing luma values 
A5AKAi+A^A7+A9y4H-Gi^3^5-<57^y4 4^ select best chroma interpolation 

Hiese predictors are composed of arithmetic averages for H i^^terpolate missing chroma values 
thechromadataandappropriatelyscaledLapladansecoDd- homontaL and vertical composite classifier 

order terms for the green data. A5N is to be used when the t^^^fi 

preferred orientation for the interpolation is in the negative 52 classiflo' test , , ^ 
diagonal direction within the pixel neighborhood. Simflarly, '^^f mterpolabon method correspondmg to the 

A5P is to be used when the preferred orientation for the ^° , f 7* "1 *u ^ * • . . ^ 
interpolation is the positive diagonal direction. A5 A is used f$ if^^^ "^^^ ^^^^ mterpolation 

when there is no clear preference for orientation for the 
interpolation. 

The coii^)lete case 3 chroma interpolation process may 

now be expressed as below. ^ £J ^^^^ 

70 avaage red/blue values 

IPDN<DP g^eea correction factor 

THEN A5 = A5N 80 block 

ELSE BP DP < DN 82 block 

THEN A5 = ASP 30 We daim: 

•p y .yp A5 = ASA 

1. Id apparatus for utilizing a digitized image signal 
obtained from an image sensor having color photositcs 
In this process both the green and the chroma data must aligned in rows and columns tiiat generate at least three 
indicate a mimmum of high spatial frequency information separate color values but only one color value for each 
for a given orientation to be chosen as the preferred orien- 35 photosite location, said apparatus including means for inter- 
tation for the interpolation. If there is a large amount of high polating the color values for each photosite location so that 
spatial frequency information in either the green data or each photosite has the three separate color values, said 
chroma data for a given orientation, it will inflate the value means for interpolating the color values comprising: 
of the corresponding classifier. This, in torn, reduces the ^^^ns for storing the digitized image signal; 

likelihood for that orientation to be chosen as the nref erred an 7 ^ 

orientation for the intemolation. ^ » ^^''"^ f^? """^ S*""" 

In practice, the case3 chroma inteipolation process my ^.ZS^l ^'^"J* ^''VJ'^'^^ 

besi«pMe<.,ashelow.forcon^„ta2^onaleffi^en.,. ^ 2 Z^^L^ pSff ^^Sa^Xr^ 

iFDN-o^DP values of different colors than the missing color value 

THEN AS = A5N ^ 3^ nearby photosite locations, said processw including 

ELSE AS a ASP means for obtaining Laplacian second-ordor values and 

gradient values in at least two image directions from 
This simpMcadon is achieved by defining the negative nearby photosites of the same column and row; 

diagonal direction as the default preferred orientation for the means for adding the Laplacian second-order values and 
interpolation when both negative and positive diagonal the gradient values to define a classifier and for select- 

classifiers are equal. The number of occurrences in a typical ing a preferred orientation for the interpolation of the 

image when the negative and positive diagonal classifiers missing color value based upon the classifier, and 

are equal is so small that this simplification generally has means for interpolating the missing color value from 
negligible impact on the image quality of the final recon- 55 nearby multiple color values selected to agree with tiic 
stracted image. preferred orientation. 

The invention has been described in detail with particular 2. In apparatus for processing a digitized image signal 
reference to certain preferred embodiments thereof, but it obtained from an image sensor having red, green, and blue 
will be understood that variations and modifications can be color photosites aligned in rows and columns that generate 
effected within the spirit and scope of the invention. ^ at least tiiree separate coIot values but only one color value 

for each photosite location, said apparatus including means 
Parts List for interpolating the color values for each photosite location 

2 input section ^ch photosite has the three separate color values, 

4 recording section means for interpolating coiiq)rising: 

10 exposure section 6S means for storing the digitized image signal; 

12 image sensor a processor operative with said storing means for gencr- 

13 color filter array ating an appropriate color value missing firom a pho- 



09/05/2003, EAST Version: 1.04.0000 



5,629,734 

9 10 

tosite location by the interpolation of an additional 8. The ^paratus of daim 1 wherein, when the preferred 

color value for such photosite locations from color orientation is negative diagonal, a color site is green, and the 

values of different coIoets than the missing color value missing color value is blue, adjacent negative diagonal blue 

at nearby photosite locations, said processor including ^^y^ y^^^^ averaged and negative diagonal green color 
means for obtaining Lapladan second-order values and 5 values including a green color site value provide a correction 

gradient values in at least two image directions from value. 

nearby photosites of the same coluiM and row^ 9. TTie apparatus of dMm 1 wherein, when the prefened 

means for addmg the Laplaaan second^orda values and ^ -^^^^^^J^L. « ^ fk*\«io«-«« 

the gradient values to define a classifier and for select- ^^^f « ^ vertical, a color ate is red, and the missmg 
inglprcfcrred orientation for the interpolation of the ,o color va^ue is green adjacen^ vemcal green color valu^^ 

missing color value based upon the classifier; and averaged and vertical red color vahics mdudmg a red color 

means for inteipolating the missing color value from site value provide a correction vahie. 
nearby multiple color values selected to agree with the 10- The apparatus of daim 1 whcrdn, when the preferred 

preferred cdentation wherein neighboring color values orientation is vertical, a color site is blue, and the missing 

of the same color are avaaged and corrected by neigh- color value is green, adjacent vertical green color values are 

boring color values of a different color value. averaged ..and vertical blue odor values including a blue 

3. The apparatus of daim 1 whcrdn, when the preferred color site value provide a correction value. 

orientation is horizontal, a color site is red, and tiie missing n. The ^paratus of daim 1 wherein, when the preferred 

color value is green, adjacent horizontal green color values orientation is vertical, a color site is green, and the missing 

arc averaged and horizontal red color values including a red color value is red, adjacent vertical red color values are 
color site value provide a correction vahie. ^ averaged and vertical green color values induding a green 

4. The apparatus of daim 1 wherein, when the prefened ^^^^ ^^^^^ provide a correction value. 

OTientation is horizontal, a color site is blue, and the missing ^ apparatus of daim 1 wherein, when the preferred 

color value is green, adjacent horizontal green col<^ values orientation is vertical, a color site is green, and the missing 
are averaged and honzootd blue coto values mdudmg a ^^^^ ^^^^^ 

blue color site value OTOvide a correction value. 25 ^ . ^ i t iij- 

5 ^^e apparatus ofdaim 1 wh^, when the prcfarcd ^^^^ vertical green cok. values indudmg a green 

orientation^ horizontal, a color site is green, and the color ate value provide a correcbion value^ 
missing color value is red, adjacent horizontal red color 13. The appaiatos of dami 1 wherem, w^^^^ 

values are averaged and horizontal green color values oaentationis positive diagonal, a color site is green, and (he 
induding a green color site value provide a correction value. 30 missing cdor value is red, adjacent positive diagonal red 

6. The apparatus of daim 1 wherein, when the prefenred color values are averaged and positive diagonal green color 
orientation is horizontal, a color site is green, and the values induding a green color site value provide a correction 
missing color value is blue, adjacent horizontal blue color value. 

values arc averaged and horizontal green color values 14. The apparatus of daim 1 wherein, when the preferred 

induding a green color site value provide a correction value. orientation is positive diagonal, a color site is green, and the 

7. The apparatus of daim 1 whcrdn, when the preferred missing color value is blue, adjacent positive diagonal blue 
orientation is negative diagonal, a cdor site is green, and the color values are averaged and positive diagonal green color 
missing color value is red, adjacent negative diagonal red values induding a green colorsite value provide a correction 
color values arc averaged and negative diagonal green color value. 

values induding a green cdor site value provide a correction 

value. ♦ If * ♦ ♦ 
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